COVNET : A cooperative coevolutionary model for evolving artificial neural networks

This paper presents COVNET, a new cooperative coevolutionary model for evolving artificial neural networks. This model is based on the idea of coevolving subnetworks. that must cooperate to form a solution for a specific problem, instead of evolving complete networks. The combination of this subnetwork is part of a coevolutionary process. The best combinations of subnetworks must be evolved together with the coevolution of the subnetworks. Several subpopulations of subnetworks coevolve cooperatively and genetically isolated. The individual of every subpopulation are combined to form whole networks. This is a different approach from most current models of evolutionary neural networks which try to develop whole networks. COVNET places as few restrictions as possible over the network structure, allowing the model to reach a wide variety of architectures during the evolution and to be easily extensible to other kind of neural networks. The performance of the model in solving three real problems of classification is compared with a modular network, the adaptive mixture of experts and with the results presented in the bibliography. COVNET has shown better generalization and produced smaller networks than the adaptive mixture of experts and has also achieved results, at least, comparable with the results in the bibliography

Self-adaptive quantum particle swarm optimization for dynamic environments

The quantum-inspired particle swarm optimization (QPSO) algorithm has been developed to find and track an optimum for dynamic optimization problems. Though QPSO has been shown to be effective, despite its simplicity, it does introduce an additional control parameter: the radius of the quantum cloud. The performance of QPSO is sensitive to the value assigned to this problem dependent parameter, which basically limits the area of the search space wherein new, better optima can be detected. This paper proposes a strategy to dynamically adapt the quantum radius, with changes in the environment. A comparison of the adaptive radius QPSO with the static radius QPSO showed that the adaptive approach achieves desirable results, without prior tuning of the quantum radius.http://link.springer.combookseries/5582019-10-03hj2018Computer Scienc

On the Bounds of Function Approximations

Within machine learning, the subfield of Neural Architecture Search (NAS) has recently garnered research attention due to its ability to improve upon human-designed models. However, the computational requirements for finding an exact solution to this problem are often intractable, and the design of the search space still requires manual intervention. In this paper we attempt to establish a formalized framework from which we can better understand the computational bounds of NAS in relation to its search space. For this, we first reformulate the function approximation problem in terms of sequences of functions, and we call it the Function Approximation (FA) problem; then we show that it is computationally infeasible to devise a procedure that solves FA for all functions to zero error, regardless of the search space. We show also that such error will be minimal if a specific class of functions is present in the search space. Subsequently, we show that machine learning as a mathematical problem is a solution strategy for FA, albeit not an effective one, and further describe a stronger version of this approach: the Approximate Architectural Search Problem (a-ASP), which is the mathematical equivalent of NAS. We leverage the framework from this paper and results from the literature to describe the conditions under which a-ASP can potentially solve FA as well as an exhaustive search, but in polynomial time.Comment: Accepted as a full paper at ICANN 2019. The final, authenticated publication will be available at https://doi.org/10.1007/978-3-030-30487-4_3

Hyper-parameter Optimisation by Restrained Stochastic Hill Climbing

Abstract. Machine learning practitioners often refer to hyper-parameter optimisation (HPO) as an art form and a skill that requires intuition and experience; Neuroevolution (NE) typically employs a combination of manual and evolutionary approaches for HPO. This paper explores the integration of a stochastic hill climbing approach for HPO within a NE algorithm. We empirically show that HPO by restrained stochastic hill climbing (HORSHC) is more effective than manual and pure evolutionary HPO. Empirical evidence is derived from a comparison of: (1) a NE algorithm that solely optimises hyper-parameters through evolution and (2) a number of derived algorithms with random search optimisation integration for optimising the hyper-parameters of a Neural Network. Through statistical analysis of the experimental results it has been revealed that random initialisation of hyper-parameters does not significantly affect the final performance of the Neural Networks evolved. However, HORSHC, a novel optimisation approach proposed in this paper has been proven to significantly out-perform the NE control algorithm. HORSHC presents itself as a solution that is computationally comparable in terms of both time and complexity as well as outperforming the control algorithm

Biochemical systems identification by a random drift particle swarm optimization approach

BACKGROUND: Finding an efficient method to solve the parameter estimation problem (inverse problem) for nonlinear biochemical dynamical systems could help promote the functional understanding at the system level for signalling pathways. The problem is stated as a data-driven nonlinear regression problem, which is converted into a nonlinear programming problem with many nonlinear differential and algebraic constraints. Due to the typical ill conditioning and multimodality nature of the problem, it is in general difficult for gradient-based local optimization methods to obtain satisfactory solutions. To surmount this limitation, many stochastic optimization methods have been employed to find the global solution of the problem. RESULTS: This paper presents an effective search strategy for a particle swarm optimization (PSO) algorithm that enhances the ability of the algorithm for estimating the parameters of complex dynamic biochemical pathways. The proposed algorithm is a new variant of random drift particle swarm optimization (RDPSO), which is used to solve the above mentioned inverse problem and compared with other well known stochastic optimization methods. Two case studies on estimating the parameters of two nonlinear biochemical dynamic models have been taken as benchmarks, under both the noise-free and noisy simulation data scenarios. CONCLUSIONS: The experimental results show that the novel variant of RDPSO algorithm is able to successfully solve the problem and obtain solutions of better quality than other global optimization methods used for finding the solution to the inverse problems in this study

An Empirical Study of Off-line Configuration and On-line Adaptation in Operator Selection

Automating the process of finding good parameter settings is important in the design of high-performing algorithms. These automatic processes can generally be categorized into off-line and on-line methods. Off-line configuration consists in learning and selecting the best setting in a training phase, and usually fixes it while solving an instance. On-line adaptation methods on the contrary vary the parameter setting adaptively during each algorithm run. In this work, we provide an empirical study of both approaches on the operator selection problem, explore the possibility of varying parameter value by a non-adaptive distribution tuned off-line, and incorporate the off-line with on-line approaches. In particular, using an off-line tuned distribution to vary parameter values at runtime appears to be a promising idea for automatic configuration. © 2014 Springer International Publishing.SCOPUS: cp.kinfo:eu-repo/semantics/publishe

Limited diversity associated with duplicated class II MHC-DRB genes in the red squirrel population in the United Kingdom compared with continental Europe

The red squirrel (Sciurus vulgaris) population in the United Kingdom has declined over the last century and is now on the UK endangered species list. This is the result of competition from the eastern grey squirrel (S. carolinensis) which was introduced in the 19th century. However, recent evidence suggests that the rate of population decline is enhanced by squirrelpox disease, caused by a viral infection carried asymptomatically by grey squirrels but to which red squirrels are highly susceptible. Population genetic diversity provides some resilience to rapidly evolving or exotic pathogens. There is currently no data on genetic diversity of extant UK squirrel populations with respect to genes involved in disease resistance. Diversity is highest at loci involved in the immune response including genes clustered within the major histocompatibility complex (MHC). Using the class II DRB locus as a marker for diversity across the MHC region we genotyped 110 red squirrels from locations in the UK and continentalEurope. Twenty four Scvu-DRB alleles at two functional loci; Scvu-DRB1 and Scvu- DRB2, were identified. High levels of diversity were identified at both loci in the continental populations. In contrast, no diversity was observed at the Scvu-DRB2 locus in the mainland UK population while a high level of homozygosity was observed at the Scvu-DRB1 locus. The red squirrel population in the UK appears to lack the extensive MHC diversity associated with continental populations, a feature which may have contributed to their rapid decline

Pleiotropy of genetic variants on obesity and smoking phenotypes: Results from the Oncoarray Project of The International Lung Cancer Consortium

Obesity and cigarette smoking are correlated through complex relationships. Common genetic causes may contribute to these correlations. In this study, we selected 241 loci potentially associated with body mass index (BMI) based on the Genetic Investigation of ANthropometric Traits (GIANT) consortium data and calculated a BMI genetic risk score (BMI-GRS) for 17,037 individuals of European descent from the Oncoarray Project of the International Lung Cancer Consortium (ILCCO). Smokers had a significantly higher BMI-GRS than never-smokers (p = 0.016 and 0.010 before and after adjustment for BMI, respectively). The BMI-GRS was also positively correlated with pack-years of smoking (p<0.001) in smokers. Based on causal network inference analyses, seven and five of 241 SNPs were classified to pleiotropic models for BMI/smoking status and BMI/pack-years, respectively. Among them, three and four SNPs associated with smoking status and pack-years (p<0.05), respectively, were followed up in the ever-smoking data of the Tobacco, Alcohol and Genetics (TAG) consortium. Among these seven candidate SNPs, one SNP (rs11030104, BDNF) achieved statistical significance after Bonferroni correction for multiple testing, and three suggestive SNPs (rs13021737, TMEM18; rs11583200, ELAVL4; and rs6990042, SGCZ) achieved a nominal statistical significance. Our results suggest that there is a common genetic component between BMI and smoking, and pleiotropy analysis can be useful to identify novel genetic loci of complex phenotypes